Augmented reality item collections

ABSTRACT

Systems and methods are provided for performing operations including: receiving, via a messaging application, input that selects a collection of augmented reality items; obtaining an identifier of the collection of the augmented reality items; searching, based on the identifier, a plurality of augmented reality items to identify a subset of augmented reality items associated with the identifier; causing the messaging application to present the subset of augmented reality items; and causing the messaging application to modify an image based on a first augmented reality item in the subset.

CLAIM OF PRIORITY

This application is a continuation of U.S. patent application Ser. No.16/834,343, filed on Mar. 30, 2020, which is incorporated herein byreference in its entirety.

TECHNICAL FIELD

The present disclosure relates generally to providing graphicalelements, such as augmented reality items, using a messagingapplication.

BACKGROUND

Modern day user devices provide messaging applications that allow usersto exchange messages with one another. Such messaging applications haverecently started incorporating graphics in such communications. Userscan select between various predetermined graphics to incorporate intotheir communications.

BRIEF DESCRIPTION OF THE DRAWINGS

In the drawings, which are not necessarily drawn to scale, like numeralsmay describe similar components in different views. To easily identifythe discussion of any particular element or act, the most significantdigit or digits in a reference number refer to the figure number inwhich that element is first introduced. Some embodiments are illustratedby way of example, and not limitation, in the figures of theaccompanying drawings in which:

FIG. 1 is a block diagram showing an example messaging system forexchanging data (e.g., messages and associated content) over a network,according to example embodiments.

FIG. 2 is a schematic diagram illustrating data which may be stored inthe database of a messaging server system, according to exampleembodiments.

FIG. 3 is a schematic diagram illustrating a structure of a messagegenerated by a messaging client application for communication, accordingto example embodiments.

FIG. 4 is a block diagram showing an example augmented reality itemcollection system, according to example embodiments.

FIG. 5 is a flowchart illustrating example operations of the augmentedreality item collection system, according to example embodiments.

FIGS. 6, 7, 8A, and 8B are illustrative inputs and outputs of theaugmented reality item collection system, according to exampleembodiments.

FIG. 9 is a block diagram illustrating a representative softwarearchitecture, which may be used in conjunction with various hardwarearchitectures herein described, according to example embodiments.

FIG. 10 is a block diagram illustrating components of a machine able toread instructions from a machine-readable medium (e.g., amachine-readable storage medium) and perform any one or more of themethodologies discussed herein, according to example embodiments.

DETAILED DESCRIPTION

The description that follows includes systems, methods, techniques,instruction sequences, and computing machine program products thatembody illustrative embodiments of the disclosure. In the followingdescription, for the purposes of explanation, numerous specific detailsare set forth in order to provide an understanding of variousembodiments. It will be evident, however, to those skilled in the art,that embodiments may be practiced without these specific details. Ingeneral, well-known instruction instances, protocols, structures, andtechniques are not necessarily shown in detail.

Typically, users exchange messages with each other using messagingapplications. Such applications allow users to select from a predefinedlist of images to send to one another. Users are increasinglycommunicating with each other using such images to convey theirthoughts. However, finding the right images to convey a particularthought can be tedious and time consuming. Specifically, the user has tomanually search using keywords for a particular image that conveys agiven message. This requires navigating through multiple pages ofinformation until the desired image is found. Given the complexity andamount of time it takes to find the right image, users becomediscouraged from communicating using the images, which results in awaste of resources or lack of use.

The disclosed embodiments improve the efficiency of using the electronicdevice by providing a system that provides navigation through augmentedreality items using different collections. The user can easily andquickly locate an augmented reality item to use to augment a capturedimage to be shared with another user in a messaging application or tovisualize how a product will look on the user to help in purchasingitems. Specifically, according to the disclosed embodiments, a messagingapplication implemented by one or more processors of a user device,receives input that selects a collection of augmented reality items. Thedisclosed embodiments obtain an identifier of the collection of theaugmented reality items and search, based on the identifier, a pluralityof augmented reality items to identify a subset of augmented realityitems associated with the identifier. The disclosed embodiments causethe messaging application to present the subset of augmented realityitems and to modify an image based on a first augmented reality item inthe subset. To assist with navigation through the augmented realityitems in the collection, the disclosed embodiments generate a displaysequence for selecting or activating different ones of the augmentedreality items in the collection. The display sequence can be conditionbased or time based.

In one example, the augmented reality items can be presented independence on a condition being satisfied, such as if a given augmentedreality item in the collection was used to enhance an image that isshared with another user or if the user is in a particular location. Inanother example, the augmented reality items can be presented andprogressed through automatically in a particular display sequence thatis specified by a third-party provider of the augmented reality items inthe collection. In such cases, one augmented reality item of acollection can be automatically presented to augment an image and, aftera certain specified threshold period of time, a second augmented realityitem in the same collection can be automatically presented to augment animage. This reduces the amount of input a user needs to provide toaccess and navigate through different augmented reality items. In thisway, a user can see how a particular lipstick color looks on their lipsin different colors by capturing an image of their face and watching howdifferent colors, corresponding to different augmented reality items ina collection of lipstick colors, are applied to the lips of their faceover a given interval. The different lipstick colors are presented andprogressed through, each being applied respectively to the lips of theuser's face, in a display sequence one after the other, each presentedfor a threshold period of time.

In this way, the disclosed embodiments improve the efficiency of usingthe electronic device by reducing the number of screens and interfaces auser has to navigate through to find an augmented reality item toaugment an image captured by the user to then share with other users orpurchase a product. This reduces the device resources (e.g., processorcycles, memory, and power usage) needed to accomplish a task with thedevice.

FIG. 1 is a block diagram showing an example messaging system 100 forexchanging data (e.g., messages and associated content) over a network106. The messaging system 100 includes multiple client devices 102, eachof which hosts a number of applications, including a messaging clientapplication 104 and a third-party application 105. Each messaging clientapplication 104 is communicatively coupled to other instances of themessaging client application 104, the third-party application 105, and amessaging server system 108 via a network 106 (e.g., the Internet).

Accordingly, each messaging client application 104 and third-partyapplication 105 is able to communicate and exchange data with anothermessaging client application 104 and third-party application(s) 105 andwith the messaging server system 108 via the network 106. The dataexchanged between messaging client applications 104, third-partyapplications 105, and between a messaging client application 104 and themessaging server system 108 includes functions (e.g., commands to invokefunctions) and payload data (e.g., text, audio, video, or othermultimedia data). Any disclosed communications between the messagingclient application 104 and the third-party application(s) 105 can betransmitted directly from the messaging client application 104 to thethird-party application(s) 105 and/or indirectly (e.g., via one or moreservers) from the messaging client application 104 to the third-partyapplication(s) 105.

The third-party application(s) 105 and the messaging client application104 are applications that include a set of functions that allow theclient device 102 to access an augmented reality item collection system124. The third-party application 105 is an application that is separateand distinct from the messaging client application 104. The third-partyapplication(s) 105 are downloaded and installed by the client device 102separately from the messaging client application 104. In someimplementations, the third-party application(s) 105 are downloaded andinstalled by the client device 102 before or after the messaging clientapplication 104 is downloaded and installed. The third-party application105 is an application that is provided by an entity or organization thatis different from the entity or organization that provides the messagingclient application 104. The third-party application 105 is anapplication that can be accessed by a client device 102 using separatelogin credentials than the messaging client application 104. Namely, thethird-party application 105 can maintain a first user account and themessaging client application 104 can maintain a second user account. Inan embodiment, the third-party application 105 can be accessed by theclient device 102 to perform various activities and interactions, suchas listening to music, videos, track exercises, view graphical elements(e.g., stickers), shop for physical items or goods, communicate withother users, and so forth.

As an example, the third-party application 105 can be a socialnetworking application, a dating application, a ride or car sharingapplication, a shopping application, a trading application, a gamingapplication, an imaging application, a music application, a videobrowsing application, an exercise tracking application, a healthmonitoring application, a graphical element or sticker browsingapplication, or any other suitable application.

The messaging client application 104 may present a menu that includes aplurality of augmented reality item collections. The messaging clientapplication 104 may receive a user selection of a first augmentedreality collection. In response, the messaging client application 104retrieves an identifier of the first augmented reality collection andsearches for a set of augmented reality items associated with theretrieved identifier. The messaging client application 104 presentsicons corresponding to the set of augmented reality items on top of alive or previous captured image or video. A given augmented reality itemmay be positioned first in the menu and visually distinguished from theremaining augmented reality items to indicate to the user that the givenaugmented reality item is currently active. The user can browse throughthe menu to select and activate a different augmented reality item.

Each augmented reality item collection includes a group of categoricallyor thematically related augmented reality items. As an example, a firstaugmented reality item collection corresponds to products available forpurchase from a first third-party entity. In this case, each augmentedreality item in the first augmented reality item collection represents adifferent product or different features (e.g., colors or styles) of asame product (e.g., lipstick or sunglasses) that can be applied to alive or previously stored image. As another example, a second augmentedreality item collection corresponds to characters (real or fictional)that are in a movie, series, video, or comic book created by a secondthird-party entity. In this case, each augmented reality item in thesecond augmented reality item collection represents a differentcharacter from the movie, series, video, or comic book that can beapplied to a live or previously stored image. As another example, athird augmented reality item collection corresponds to popular retroeffects or visual effects created by a given third-party entity. In thiscase, each augmented reality item in the second augmented reality itemcollection represents a different popular retro effect or visual effectthat can be applied to a live or previously stored image. As anotherexample, a fourth augmented reality item collection corresponds toflower crowns. In this case, each augmented reality item in the secondaugmented reality item collection represents a different flower crownthat can be applied to a person's head depicted in a live or previouslystored image.

To increase the overall efficiency and reduce bandwidth consumed by aclient device 102, the messaging client application 104 may only loadthe graphical elements for a subset of the augmented reality items thatare presented. For example, if a given augmented reality item collectionincludes 10 different augmented reality items, the messaging clientapplication 104 retrieves from a server the graphical elements, metadataand information for augmenting an image or video using the augmentedreality items for the first three of the 10 different augmented realityitems. The remaining seven augmented reality items may be represented inthe display using respective icons that may be greyed out. When a userinput, such as tapping a given one of the seven icons, is received thatselects a given augmented reality item for which the graphical elements,metadata and information has not yet been loaded, the messaging clientapplication 104 then communicates with the server to retrieve thegraphical elements, metadata and information.

The graphical elements of the active augmented reality item areretrieved and used to augment the image or video presented by themessaging client application 104. For example, the active augmentedreality item corresponds to a lipstick color. In such cases, themessaging client application 104 identifies the lips region of a facedepicted in the image or video and modifies the color of the lips regionbased on the color of the graphical elements corresponding to the activeaugmented reality item.

The menu of augmented reality collections may include identifiers (e.g.,thumbnails) that represent each augmented reality item collection. Theidentifiers may be arranged based on index positions associated witheach augmented reality item collection. Augmented reality itemcollections with low index numbers are presented earlier in the menuthan augmented reality item collections with higher index numbers. Asone augmented reality item collection is removed from the menu, thoseaugmented reality items that have higher index numbers are advanced inthe menu and have their respective index numbers reduced by one. Certainaugmented reality item collections are associated with active periods.The active periods may include start and end times or just start times.When the current time is past the start time and, in certain cases whenan end time is present, before the end time, the augmented reality itemcollection enters the active period. During the active period, theaugmented reality item collection can be selected by the user to access,view and interact with the augmented reality items associated with thecollection. During the active period, the indicator for the activeaugmented reality item becomes visible in the menu (e.g., not greyed outor added to the menu at a particular position corresponding to the indexvalue of the augmented reality item collection). During an inactiveperiod of an augmented reality item collection, such as when the currenttime is before the start time or is after the end time of the activeperiod, the indicator for the augmented reality item collection isremoved from the menu or is greyed out.

In some cases, the messaging client application 104 may present a menuthat includes a plurality of augmented reality items. The messagingclient application 104 may receive a user selection of a first augmentedreality item. In response, the messaging client application 104retrieves an identifier of a collection associated with the firstaugmented reality item and searches for a set of augmented reality itemsassociated with the retrieved identifier of the collection. Themessaging client application 104 presents icons corresponding to the setof augmented reality items on top of a live or previous captured imageor video. The selected first augmented reality item is visuallydistinguished from the remaining augmented reality items to indicate tothe user that the first augmented reality item is currently active. Theuser can browse through the menu to select and activate a differentaugmented reality item of the collection corresponding to the firstaugmented reality item. The graphical elements of the active augmentedreality item are retrieved and used to augment the image or videopresented by the messaging client application 104. For example, thefirst augmented reality item corresponds to a first superhero. In suchcases, the messaging client application 104 identifies a collection ofsuperheroes associated with the first superhero and presents icons ofall of the superheroes in the collection including the first superhero.The messaging client application 104 modifies a look and feel (bodysize, style and proportion) of a bode depicted in the image or videobased on attributes of the graphical elements corresponding to the firstsuperhero. As a result, the person depicted in the image looks and feelslike the first superhero.

The messaging client application 104 may automatically progress throughthe augmented reality items in the selected collection that arepresented according to a particular display or presentation sequence.Each augmented reality item in the collection includes metadata thatindicates position information. The messaging client application 104 mayuse the position information to determine when a given augmented realityitem is automatically activated and which augmented reality item topresent next in the display sequence. For example, the messaging clientapplication 104 may retrieve the position information of each of theaugmented reality items that are in a collection. The messaging clientapplication 104 may form a group that includes some or all of theaugmented reality items (e.g., the group may include only 3 augmentedreality items of the 15 that are in the collection). The group mayinclude those augmented reality items that have a position number orpresentation order that exceeds a threshold. The display sequence of theaugmented reality items in the group may be different from the displaysequential arrangement of the augmented reality items in the collectionof augmented reality items. The messaging client application 104 mayarrange the group of augmented reality items in a display sequence basedon their position numbers or presentation order. The messaging clientapplication 104 may then automatically start applying each augmentedreality item in the display sequence to an image or video displayed bythe messaging client application 104.

The messaging client application 104 may present the first augmentedreality item to the image or video that is positioned first in thedisplay sequence. After a threshold period of time, such as after 4seconds, the messaging client application 104 automatically activatesand presents a second augmented reality item that is adjacent to thefirst augmented reality item in the display sequence or presentationorder. The graphical elements of the first augmented reality item areremoved from presentation and replaced by graphical elements of thesecond augmented reality item. When the last augmented reality item inthe presentation order is presented for the threshold period of time,the messaging client application 104 may return to activating the firstaugmented reality item. Alternatively, the messaging client application104 may inform the user that all of the augmented reality items in thegroup have been presented and request that the user confirm that theuser wants to repeat the automatic presentation of the group ofaugmented reality items.

In one example, the messaging client application 104 presents a firstaugmented reality item in the collection and obtains metadata associatedwith the first augmented reality item. The messaging client application104 determines based on the metadata that a condition needs to besatisfied for presenting a second augmented reality item in thecollection. For example, the condition may be event-based,location-based or temporal-based. In particular, the event-basedcondition may specify that the condition is satisfied when an imageaugmented with the first augmented reality items is shared with anotheruser. In response to determining that the user augmented an image orvideo using the first augmented reality item and then sent the augmentedimage or video to a friend, the condition specified in the metadata forthe first augmented reality item is marked as satisfied. In such cases,the second augmented reality items becomes activated and selectable in amenu of the augmented reality items in the collection. The secondaugmented reality item may include metadata with another condition thatneeds to be satisfied for presenting and activating a third augmentedreality item in the collection. The condition specified in the metadatafor the second augmented reality item may be location-based. Such acondition is satisfied when a current location of the client device 102is within a threshold distance of the location indicated in thecondition. In response to determining that the client device 102 iswithin the threshold distance of the location indicated in the metadata,the condition specified in the metadata for the second augmented realityitem is marked as satisfied. In such cases, the third augmented realityitems becomes activated and selectable in a menu of the augmentedreality items in the collection.

The condition specified in the metadata for the third augmented realityitem may be event-based and associated with a different event than theevent-based condition for the first augmented reality item. Theevent-based condition for the third augmented reality item may besatisfied if the client device 102 captures content (e.g., an image orvideo) that includes a specific object associated with the collection.For example, the third augmented reality item may be a shoe provided bya third-party entity. When the user captures an image of anadvertisement for the same shoe provided by the third-party entity, thecondition is satisfied and a fourth augmented reality item in thecollection may be activated and presented or selectable from the menu.

The fourth augmented reality item may be associated with a time-basedcondition. The time-based condition may include a start time and an endtime for enabling access to the fourth augmented reality item. If thetime-based condition is satisfied (e.g., when the current time isbetween the start time and end time of the fourth augmented realityitem), the fourth augmented reality item becomes selectable andactivated. If the time-based condition is not satisfied, a message orprompt may be presented to the user informing the user to come back soonat the start time to access the fourth augmented reality item.

The messaging server system 108 provides server-side functionality viathe network 106 to a particular messaging client application 104. Whilecertain functions of the messaging system 100 are described herein asbeing performed by either a messaging client application 104 or by themessaging server system 108, it will be appreciated that the location ofcertain functionality either within the messaging client application 104or the messaging server system 108 is a design choice. For example, itmay be technically preferable to initially deploy certain technology andfunctionality within the messaging server system 108, but to latermigrate this technology and functionality to the messaging clientapplication 104 where a client device 102 has a sufficient processingcapacity.

The messaging server system 108 supports various services and operationsthat are provided to the messaging client application 104. Suchoperations include transmitting data to, receiving data from, andprocessing data generated by the messaging client application 104. Thisdata may include message content, client device information, graphicalelements, geolocation information, media annotation and overlays,virtual objects, message content persistence conditions, social networkinformation, and live event information, as examples. Data exchangeswithin the messaging system 100 are invoked and controlled throughfunctions available via user interfaces (UIs) (e.g., graphical userinterfaces) of the messaging client application 104.

Turning now specifically to the messaging server system 108, an APIserver 110 is coupled to, and provides a programmatic interface to, anapplication server 112. The application server 112 is communicativelycoupled to a database server 118, which facilitates access to a database120 in which is stored data associated with messages processed by theapplication server 112.

Dealing specifically with the API server 110, this server 110 receivesand transmits message data (e.g., commands and message payloads) betweenthe client device 102 and the application server 112. Specifically, theAPI server 110 provides a set of interfaces (e.g., routines andprotocols) that can be called or queried by the messaging clientapplication 104 and the third-party application 105 in order to invokefunctionality of the application server 112. The API server 110 exposesvarious functions supported by the application server 112, includingaccount registration; login functionality; the sending of messages, viathe application server 112, from a particular messaging clientapplication 104 to another messaging client application 104 orthird-party application 105; the sending of media files (e.g., graphicalelements, images or video) from a messaging client application 104 to amessaging server application 114, and for possible access by anothermessaging client application 104 or third-party application 105; agraphical element list; the setting of a collection of media data (e.g.,story); the retrieval of such collections; the retrieval of a list offriends of a user of a client device 102; maintaining augmented realityitems; the retrieval of messages and content; the adding and deleting offriends to a social graph; the location of friends within a socialgraph; access to user conversation data; access to avatar informationstored on messaging server system 108; and opening an application event(e.g., relating to the messaging client application 104).

The application server 112 hosts a number of applications andsubsystems, including a messaging server application 114, an imageprocessing system 116, a social network system 122, and the augmentedreality item collection system 124. The messaging server application 114implements a number of message processing technologies and functions,particularly related to the aggregation and other processing of content(e.g., textual and multimedia content) included in messages receivedfrom multiple instances of the messaging client application 104. As willbe described in further detail, the text and media content from multiplesources may be aggregated into collections of content (e.g., calledstories or galleries). These collections are then made available, by themessaging server application 114, to the messaging client application104. Other processor- and memory-intensive processing of data may alsobe performed server-side by the messaging server application 114, inview of the hardware requirements for such processing.

The application server 112 also includes an image processing system 116that is dedicated to performing various image processing operations,typically with respect to images or video received within the payload ofa message at the messaging server application 114. A portion of theimage processing system 116 may also be implemented by the augmentedreality item collection system 124.

The social network system 122 supports various social networkingfunctions and services and makes these functions and services availableto the messaging server application 114. To this end, the social networksystem 122 maintains and accesses an entity graph within the database120. Examples of functions and services supported by the social networksystem 122 include the identification of other users of the messagingsystem 100 with which a particular user has relationships or is“following” and also the identification of other entities and interestsof a particular user. Such other users may be referred to as the user'sfriends. Social network system 122 may access location informationassociated with each of the user's friends to determine where they liveor are currently located geographically. Social network system 122 maymaintain a location profile for each of the user's friends indicatingthe geographical location where the user's friends live.

The augmented reality item collection system 124 manages augmentedreality item collections. The augmented reality item collection system124 enables third-party entities to submit augmented reality items andassign such augmented reality items to one or more augmented realityitem collections. The third-party entities can generate an augmentedreality item collection if they provide three or more augmented realityitems. The augmented reality item collection system 124 publishes theaugmented reality item collections and presents the augmented realityitem collections to users in a menu. The menu presents indicators (e.g.,icons) representing the augmented reality item collections to a user andarranges such icons based on metadata (e.g., index or positioninformation) associated with the augmented reality item collections. Theaugmented reality item collection system 124 receives input from a userthat selects a given augmented reality items collection and searches foraugmented reality items associated with a unique or specific identifierof the augmented reality item collection. The augmented reality itemcollection system 124 presents the identified augmented reality itemsresulting from the search to the user to enable the user to augmented animage or video using the augmented reality items. In some cases, theaugmented reality item collection system 124 automatically progressesthrough the identified (or subset of the identified) augmented realityitems, such that one augmented reality item is presented after anotheraugmented reality item automatically after a threshold period of time(e.g., 4 seconds).

The application server 112 is communicatively coupled to a databaseserver 118, which facilitates access to a database 120, in which isstored data associated with messages processed by the messaging serverapplication 114. Database 120 may be a third-party database. Forexample, the application server 112 may be associated with a firstentity, and the database 120 or a portion of the database 120 may beassociated and hosted by a second different entity. In someimplementations, database 120 stores user data that the first entitycollects about various each of the users of a service provided by thefirst entity. For example, the user data includes user names, phonenumbers, passwords, addresses, friends, activity information,preferences, videos or content consumed by the user, and so forth.

FIG. 2 is a schematic diagram 200 illustrating data, which may be storedin the database 120 of the messaging server system 108, according tocertain example embodiments. While the content of the database 120 isshown to comprise a number of tables, it will be appreciated that thedata could be stored in other types of data structures (e.g., as anobject-oriented database).

The database 120 includes message data stored within a message table214. An entity table 202 stores entity data, including an entity graph204. Entities for which records are maintained within the entity table202 may include individuals, corporate entities, organizations, objects,places, events, and so forth. Regardless of type, any entity regardingwhich the messaging server system 108 stores data may be a recognizedentity. Each entity is provided with a unique identifier, as well as anentity type identifier (not shown).

The entity graph 204 stores information regarding relationships andassociations between entities. Such relationships may be social,professional (e.g., work at a common corporation or organization),interest-based, or activity-based, merely for example.

Message table 214 may store a collection of conversations between a userand one or more friends or entities. Message table 214 may includevarious attributes of each conversation, such as the list ofparticipants, the size of the conversation (e.g., number of users and/ornumber of messages), the chat color of the conversation, a uniqueidentifier for the conversation, and any other conversation relatedfeature(s).

The database 120 also stores annotation data, in the example form offilters, in an annotation table 212. Database 120 also stores annotatedcontent received in the annotation table 212. Filters for which data isstored within the annotation table 212 are associated with and appliedto videos (for which data is stored in a video table 210) and/or images(for which data is stored in an image table 208). Filters, in oneexample, are overlays that are displayed as overlaid on an image orvideo during presentation to a recipient user. Filters may be of varioustypes, including user-selected filters from a gallery of filterspresented to a sending user by the messaging client application 104 whenthe sending user is composing a message. Other types of filters includegeolocation filters (also known as geo-filters), which may be presentedto a sending user based on geographic location. For example, geolocationfilters specific to a neighborhood or special location may be presentedwithin a UI by the messaging client application 104, based ongeolocation information determined by a Global Positioning System (GPS)unit of the client device 102. Another type of filter is a data filter,which may be selectively presented to a sending user by the messagingclient application 104, based on other inputs or information gathered bythe client device 102 during the message creation process. Examples ofdata filters include current temperature at a specific location, acurrent speed at which a sending user is traveling, battery life for aclient device 102, or the current time.

Other annotation data that may be stored within the image table 208 isso-called “LENS” data. A “LENS” may be a real-time special effect andsound that may be added to an image or a video. A LENS is also referredto as an augmented reality item.

As described above, LENSES, overlays, image transformations, AR imagesand similar terms refer to modifications that may be made to videos orimages. This includes real-time modification which modifies an image asit is captured using a device sensor and then displayed on a screen ofthe device with the modifications. This also includes modifications tostored content, such as video clips in a gallery that may be modified.For example, in a device with access to multiple LENSES, a user can usea single video clip with multiple LENSES to see how the different LENSESwill modify the stored clip. For example, multiple LENSES that applydifferent pseudorandom movement models can be applied to the samecontent by selecting different LENSES for the content. Similarly,real-time video capture may be used with an illustrated modification toshow how video images currently being captured by sensors of a devicewould modify the captured data. Such data may simply be displayed on thescreen and not stored in memory, or the content captured by the devicesensors may be recorded and stored in memory with or without themodifications (or both). In some systems, a preview feature can show howdifferent LENSES will look within different windows in a display at thesame time. This can, for example, enable multiple windows with differentpseudorandom animations to be viewed on a display at the same time.

Data and various systems to use LENSES or other such transform systemsto modify content using this data can thus involve detection of objects(e.g. faces, hands, bodies, cats, dogs, surfaces, objects, etc.),tracking of such objects as they leave, enter, and move around the fieldof view in video frames, and the modification or transformation of suchobjects as they are tracked. In various embodiments, different methodsfor achieving such transformations may be used. For example, someembodiments may involve generating a three-dimensional mesh model of theobject or objects, and using transformations and animated textures ofthe model within the video to achieve the transformation. In otherembodiments, tracking of points on an object may be used to place animage or texture (which may be two dimensional or three dimensional) atthe tracked position. In still further embodiments, neural networkanalysis of video frames may be used to place images, models, ortextures in content (e.g. images or frames of video). LENS data thusrefers both to the images, models, and textures used to createtransformations in content, as well as to additional modeling andanalysis information needed to achieve such transformations with objectdetection, tracking, and placement.

Real time video processing can be performed with any kind of video data,(e.g. video streams, video files, etc.) saved in a memory of acomputerized system of any kind. For example, a user can load videofiles and save them in a memory of a device, or can generate a videostream using sensors of the device. Additionally, any objects can beprocessed using a computer animation model, such as a human's face andparts of a human body, animals, or non-living things such as chairs,cars, or other objects.

In some embodiments, when a particular modification is selected alongwith content to be transformed, elements to be transformed areidentified by the computing device, and then detected and tracked ifthey are present in the frames of the video. The elements of the objectare modified according to the request for modification, thustransforming the frames of the video stream. Transformation of frames ofa video stream can be performed by different methods for different kindsof transformation. For example, for transformations of frames mostlyreferring to changing forms of object's elements characteristic pointsfor each of element of an object are calculated (e.g. using an ActiveShape Model (ASM) or other known methods). Then, a mesh based on thecharacteristic points is generated for each of the at least one elementof the object. This mesh used in the following stage of tracking theelements of the object in the video stream. In the process of tracking,the mentioned mesh for each element is aligned with a position of eachelement. Then, additional points are generated on the mesh. A first setof first points is generated for each element based on a request formodification, and a set of second points is generated for each elementbased on the set of first points and the request for modification. Then,the frames of the video stream can be transformed by modifying theelements of the object on the basis of the sets of first and secondpoints and the mesh. In such method a background of the modified objectcan be changed or distorted as well by tracking and modifying thebackground.

In one or more embodiments, transformations changing some areas of anobject using its elements can be performed by calculating ofcharacteristic points for each element of an object and generating amesh based on the calculated characteristic points. Points are generatedon the mesh, and then various areas based on the points are generated.The elements of the object are then tracked by aligning the area foreach element with a position for each of the at least one element, andproperties of the areas can be modified based on the request formodification, thus transforming the frames of the video stream.Depending on the specific request for modification properties of thementioned areas can be transformed in different ways. Such modificationsmay involve: changing color of areas; removing at least some part ofareas from the frames of the video stream; including one or more newobjects into areas which are based on a request for modification; andmodifying or distorting the elements of an area or object. In variousembodiments, any combination of such modifications or other similarmodifications may be used. For certain models to be animated, somecharacteristic points can be selected as control points to be used indetermining the entire state-space of options for the model animation.

In some embodiments of a computer animation model to transform imagedata using face detection, the face is detected on an image with use ofa specific face detection algorithm (e.g. Viola-Jones). Then, an ActiveShape Model (ASM) algorithm is applied to the face region of an image todetect facial feature reference points.

In other embodiments, other methods and algorithms suitable for facedetection can be used. For example, in some embodiments, features arelocated using a landmark which represents a distinguishable pointpresent in most of the images under consideration. For facial landmarks,for example, the location of the left eye pupil may be used. In aninitial landmark is not identifiable (e.g. if a person has an eyepatch),secondary landmarks may be used. Such landmark identification proceduresmay be used for any such objects. In some embodiments, a set oflandmarks forms a shape. Shapes can be represented as vectors using thecoordinates of the points in the shape. One shape is aligned to anotherwith a similarity transform (allowing translation, scaling, androtation) that minimizes the average Euclidean distance between shapepoints. The mean shape is the mean of the aligned training shapes.

In some embodiments, a search for landmarks from the mean shape alignedto the position and size of the face determined by a global facedetector is started. Such a search then repeats the steps of suggestinga tentative shape by adjusting the locations of shape points by templatematching of the image texture around each point and then conforming thetentative shape to a global shape model until convergence occurs. Insome systems, individual template matches are unreliable and the shapemodel pools the results of the weak template matchers to form a strongeroverall classifier. The entire search is repeated at each level in animage pyramid, from coarse to fine resolution.

Embodiments of a transformation system can capture an image or videostream on a client device and perform complex image manipulationslocally on a client device such as client device 102 while maintaining asuitable user experience, computation time, and power consumption. Thecomplex image manipulations may include size and shape changes, emotiontransfers (e.g., changing a face from a frown to a smile), statetransfers (e.g., aging a subject, reducing apparent age, changinggender), style transfers, graphical element application, and any othersuitable image or video manipulation implemented by a convolutionalneural network that has been configured to execute efficiently on aclient device.

In some example embodiments, a computer animation model to transformimage data can be used by a system where a user may capture an image orvideo stream of the user (e.g., a selfie) using a client device 102having a neural network operating as part of a messaging application 104operating on the client device 102. The transform system operatingwithin the messaging application 104 determines the presence of a facewithin the image or video stream and provides modification iconsassociated with a computer animation model to transform image data, orthe computer animation model can be present as associated with aninterface described herein. The modification icons include changes whichmay be the basis for modifying the user's face within the image or videostream as part of the modification operation. Once a modification iconis selected, the transform system initiates a process to convert theimage of the user to reflect the selected modification icon (e.g.,generate a smiling face on the user). In some embodiments, a modifiedimage or video stream may be presented in a graphical user interfacedisplayed on the mobile client device as soon as the image or videostream is captured and a specified modification is selected. Thetransform system may implement a complex convolutional neural network ona portion of the image or video stream to generate and apply theselected modification. That is, the user may capture the image or videostream and be presented with a modified result in real time or near realtime once a modification icon has been selected. Further, themodification may be persistent while the video stream is being capturedand the selected modification icon remains toggled. Machine taughtneural networks may be used to enable such modifications.

In some embodiments, the graphical user interface, presenting themodification performed by the transform system, may supply the user withadditional interaction options. Such options may be based on theinterface used to initiate the content capture and selection of aparticular computer animation model (e.g. initiation from a contentcreator user interface). In various embodiments, a modification may bepersistent after an initial selection of a modification icon. The usermay toggle the modification on or off by tapping or otherwise selectingthe face being modified by the transformation system and store it forlater viewing or browse to other areas of the imaging application. Wheremultiple faces are modified by the transformation system, the user maytoggle the modification on or off globally by tapping or selecting asingle face modified and displayed within a graphical user interface. Insome embodiments, individual faces, among a group of multiple faces, maybe individually modified or such modifications may be individuallytoggled by tapping or selecting the individual face or a series ofindividual faces displayed within the graphical user interface.

As mentioned above, the video table 210 stores video data which, in oneembodiment, is associated with messages for which records are maintainedwithin the message table 214. Similarly, the image table 208 storesimage data associated with messages for which message data is stored inthe entity table 202. The entity table 202 may associate variousannotations from the annotation table 212 with various images and videosstored in the image table 208 and the video table 210.

Augmented reality item list 207 stores augmented reality items availablefor selection and activation by the messaging client application 104.Each augmented reality item in the augmented reality item list 207 isassociated with one or more collections. In an embodiment, the messagingclient application 104 searches the collection identifiers stored in theaugmented reality item list 207 to identify one or more augmentedreality items associated with an augmented reality item collection thatis selected. Each augmented reality item stored in the augmented realityitem list 207 includes one or more graphical elements or virtual objectswhich may or may not be animated. Each augmented reality item alsoincludes instructions on where to position the graphical elements orvirtual objects relative to other objects depicted in the capturedimage. Certain augmented reality items are associated with purchasableproducts. Such augmented reality items include a product catalogidentifier and a product identifier. These augmented reality items aredynamically modifiable such that, when the augmented reality items areactivated, the product catalog is accessed based on the product catalogidentifier. A current price of the product is retrieved from the productcatalog using the product identifier as well as updated graphicalelement information (e.g., position, color, size, style, and so forth).The current price may be presented concurrently with the activeaugmented reality item and a user can select an option to automaticallypurchase the product associated with the augmented reality item that isactive.

A story table 206 stores data regarding collections of messages andassociated image, video, or audio data, which are compiled into acollection (e.g., a story or a gallery). The creation of a particularcollection may be initiated by a particular user (e.g., each user forwhich a record is maintained in the entity table 202). A user may createa “personal story” in the form of a collection of content that has beencreated and sent/broadcast by that user. To this end, the UI of themessaging client application 104 may include an icon that isuser-selectable to enable a sending user to add specific content to hisor her personal story.

A collection may also constitute a “live story,” which is a collectionof content from multiple users that is created manually, automatically,or using a combination of manual and automatic techniques. For example,a “live story” may constitute a curated stream of user-submitted contentfrom various locations and events. Users whose client devices havelocation services enabled and are at a common location event at aparticular time may, for example, be presented with an option, via a UIof the messaging client application 104, to contribute content to aparticular live story. The live story may be identified to the user bythe messaging client application 104 based on his or her location. Theend result is a “live story” told from a community perspective.

A further type of content collection is known as a “location story,”which enables a user whose client device 102 is located within aspecific geographic location (e.g., on a college or university campus)to contribute to a particular collection. In some embodiments, acontribution to a location story may require a second degree ofauthentication to verify that the end user belongs to a specificorganization or other entity (e.g., is a student on the universitycampus).

FIG. 3 is a schematic diagram illustrating a structure of a message 300,according to some embodiments, generated by a messaging clientapplication 104 for communication to a further messaging clientapplication 104 or the messaging server application 114. The content ofa particular message 300 is used to populate the message table 214stored within the database 120, accessible by the messaging serverapplication 114. Similarly, the content of a message 300 is stored inmemory as “in-transit” or “in-flight” data of the client device 102 orthe application server 112. The message 300 is shown to include thefollowing components:

-   -   A message identifier 302: a unique identifier that identifies        the message 300.    -   A message text payload 304: text, to be generated by a user via        a UI of the client device 102 and that is included in the        message 300.    -   A message image payload 306: image data, captured by a camera        component of a client device 102 or retrieved from memory of a        client device 102, and that is included in the message 300.    -   A message video payload 308: video data, captured by a camera        component or retrieved from a memory component of the client        device 102 and that is included in the message 300.    -   A message audio payload 310: audio data, captured by a        microphone or retrieved from the memory component of the client        device 102, and that is included in the message 300.    -   Message annotations 312: annotation data (e.g., filters,        stickers, or other enhancements) that represents annotations to        be applied to message image payload 306, message video payload        308, or message audio payload 310 of the message 300.    -   A message duration parameter 314: parameter value indicating, in        seconds, the amount of time for which content of the message        (e.g., the message image payload 306, message video payload 308,        message audio payload 310) is to be presented or made accessible        to a user via the messaging client application 104.    -   A message geolocation parameter 316: geolocation data (e.g.,        latitudinal and longitudinal coordinates) associated with the        content payload of the message. Multiple message geolocation        parameter 316 values may be included in the payload, with each        of these parameter values being associated with respect to        content items included in the content (e.g., a specific image        within the message image payload 306, or a specific video in the        message video payload 308).    -   A message story identifier 318: identifier value identifying one        or more content collections (e.g., “stories”) with which a        particular content item in the message image payload 306 of the        message 300 is associated. For example, multiple images within        the message image payload 306 may each be associated with        multiple content collections using identifier values.    -   A message tag 320: each message 300 may be tagged with multiple        tags, each of which is indicative of the subject matter of        content included in the message payload. For example, where a        particular image included in the message image payload 306        depicts an animal (e.g., a lion), a tag value may be included        within the message tag 320 that is indicative of the relevant        animal. Tag values may be generated manually, based on user        input, or may be automatically generated using, for example,        image recognition.    -   A message sender identifier 322: an identifier (e.g., a        messaging system identifier, email address, or device        identifier) indicative of a user of the client device 102 on        which the message 300 was generated and from which the message        300 was sent.    -   A message receiver identifier 324: an identifier (e.g., a        messaging system identifier, email address, or device        identifier) indicative of user(s) of the client device 102 to        which the message 300 is addressed. In the case of a        conversation between multiple users, the identifier may indicate        each user involved in the conversation.

The contents (e.g., values) of the various components of message 300 maybe pointers to locations in tables within which content data values arestored. For example, an image value in the message image payload 306 maybe a pointer to (or address of) a location within an image table 208.Similarly, values within the message video payload 308 may point to datastored within a video table 210, values stored within the messageannotations 312 may point to data stored in an annotation table 212,values stored within the message story identifier 318 may point to datastored in a story table 206, and values stored within the message senderidentifier 322 and the message receiver identifier 324 may point to userrecords stored within an entity table 202.

FIG. 4 is a block diagram showing an example augmented reality itemcollection system 124, according to example embodiments. Augmentedreality item collection system 124 includes collection selection module414, an augmented reality item identification module 416, and anaugmented reality item presentation module 418.

The collection selection module 414 receives input from a user thatselects a given augmented reality item collection. As an example, theuser can navigate through a menu of augmented reality item collectionsand tap or select a given one of the augmented reality items collectionsto select the augmented reality item collection. As another example, theuser can navigate through a menu of individual augmented reality items.In response to the user tapping or selecting a given augmented realityitem from the menu, the augmented reality collection associated with thegiven augmented reality item is identified and used to retrieve aremaining set of the augmented reality items associated with the samecollection. The collection selection module 414 retrieves an augmentedreality item collection identifier for the given augmented reality itemcollection.

The augmented reality item identification module 416 receives theaugmented reality item collection identifier from the collectionselection module 414. The augmented reality item identification module416 searches the augmented reality item list 207 to identify one or moreaugmented reality items that are associated with the collectionidentifier. For example, the collection identifier may indicate that theselected collection corresponds to a lipstick provided by a third-partyentity relative to the provider of the messaging client application 104.In such cases, the augmented reality item identification module 416identifies a set of augmented reality items that are associated withthat lipstick. For example, the set of augmented reality items includesa set of graphical elements that modify lips of a face depicted in animage using different colors to represent different colors of thelipstick. In another example, the collection identifier may indicatethat the selected collection corresponds to a collection of superheroes.In such cases, the augmented reality item identification module 416identifies a set of augmented reality items that are associated with thesuperheroes. For example, the set of augmented reality items includes aset of graphical elements that modify a person depicted in an image tolook like different superheroes corresponding to respective ones of theset of augmented reality items.

The augmented reality item presentation module 418 receives the list ofaugmented reality items associated with the augmented reality itemcollection. The augmented reality item presentation module 418 presentsa menu on the messaging client application 104 that enables the user tonavigate through and active augmented reality items that are part of theaugmented reality item collection. The augmented reality itempresentation module 418 may selectively load only a first portion of theaugmented reality items that are represented in the menu. For example,the augmented reality item presentation module 418 may only load (e.g.,retrieve from a server) the graphical elements, metadata and informationassociated with a first portion of the augmented reality items that aredisplayed in the menu. When the user selects an augmented reality itemfor which data has not been loaded, then the augmented reality itempresentation module 418 retrieves from the server only the graphicalelements, metadata and information for the selected augmented realityitem or for the selected augmented reality item and a second portion(e.g., the next three adjacent augmented reality items) in the menu.

The augmented reality item presentation module 418 selectively activatesor enables for selection augmented reality items from the menu based onconditions. For example, the augmented reality item presentation module418 may wait for event-based, location-based or time-based conditionsassociated with a first augmented reality item or first group ofaugmented reality items to be satisfied before activating a secondaugmented reality item or second group of augmented reality items.

FIG. 5 is a flowchart illustrating example operations of the augmentedreality item collection system 124 in performing process 500, accordingto example embodiments. The process 500 may be embodied incomputer-readable instructions for execution by one or more processorssuch that the operations of the process 500 may be performed in part orin whole by the functional components of the messaging server system 108and/or third-party application 105; accordingly, the process 500 isdescribed below by way of example with reference thereto. However, inother embodiments, at least some of the operations of the process 500may be deployed on various other hardware configurations. The process500 is therefore not intended to be limited to the messaging serversystem 108 and can be implemented in whole, or in part, by any othercomponent. Some or all of the operations of process 500 can be inparallel, out of order, or entirely omitted.

At operation 501, the augmented reality item collection system 124receives (e.g., via a messaging application installed on a client device102) input that selects a collection of augmented reality items. As anexample, the user can navigate through a menu of augmented reality itemcollections displayed by the messaging client application 104 and tap orselect a given one of the augmented reality items collections to selectthe augmented reality item collection. As another example, the user cannavigate through a menu of individual augmented reality items. Inresponse to the user tapping or selecting a given augmented reality itemfrom the menu, the augmented reality collection associated with thegiven augmented reality item is identified and used to retrieve aremaining set of the augmented reality items associated with the samecollection. The collection selection module 414 retrieves an augmentedreality item collection identifier for the given augmented reality itemcollection.

At operation 502, the augmented reality item collection system 124obtains an identifier of the collection of the augmented reality items.For example, the augmented reality item collection system 124 accessesmetadata for the selected augmented reality item collection stored inaugmented reality item list 207 to retrieve a unique identifier (e.g.,name or serial number) of the augmented reality item collection.

At operation 503, the augmented reality item collection system 124searches, based on the identifier, a plurality of augmented realityitems to identify a subset of augmented reality items associated withthe identifier. For example, the augmented reality item identificationmodule 416 searches the augmented reality item list 207 to identify oneor more augmented reality items that are associated with the collectionidentifier.

At operation 504, the augmented reality item collection system 124causes the messaging application to present the subset of augmentedreality items. For example, the augmented reality item presentationmodule 418 presents a menu on the messaging client application 104 thatenables the user to navigate through and active augmented reality itemsthat are part of the augmented reality item collection.

At operation 505, the augmented reality item collection system 124causes the messaging application to modify an image based on a firstaugmented reality item in the subset. For example, the collectionidentifier may indicate that the selected collection corresponds to alipstick provided by a third-party entity relative to the provider ofthe messaging client application 104. In such cases, the augmentedreality item identification module 416 identifies a set of augmentedreality items that are associated with that lipstick. For example, theset of augmented reality items includes a set of graphical elements thatmodify lips of a face depicted in an image using different colors torepresent different colors of the lipstick.

FIGS. 6-8 are illustrative inputs and outputs of the augmented realityitem collection system 124, according to example embodiments. Theaugmented reality item collection system 124 presents a user interface600 that allows a third-party entity to create, modify, or deleteaugmented reality items and augmented reality item collections. Userinterface 600 lists various augmented reality item collections that havebeen created. An option 602 can be selected to create a new augmentedreality item collection. A first augmented reality item collection 601is positioned first in the list of augmented reality item collectionsbased on an index or position number assigned to the augmented realityitem collection 601. In response to receiving a user selection of option602, user interface 610 is presented to allow a user to add a newaugmented reality item collection.

User interface 610 has various input fields for defining parameters ofthe augmented reality item collection. The input fields include a namefield in which a name can be assigned to the augmented reality itemcollection, a position field which can be used to assign an index orposition to the augmented reality item collection, details that identifythe third-party entity fields, and a thumbnail field that can be used tosupply an image to represent the augmented reality item with anindicator in a menu for selection. User interface 610 may includeportions of user interface 620 which include scheduling informationfields that enable a user to define an activation period (e.g., startingtime and ending time or just a starting time) for the augmented realityitem collection. User interface 620 may list augmented reality items 621associated with the augmented reality item collection. An option 622 isprovided that allows the user to add a new augmented reality item to thecollection or specify previously created augmented reality item toassociated with this augmented reality item collection. The user canadjust the positions of the augmented reality items that are presentedto a user in a menu when the user selects the collection by assigningdifferent position or index numbers to the augmented reality items 621associated with the collection. The user can also drag and drop variousones of the augmented reality items 621 into respective positions tochange the display or presentation order of the augmented reality items621 that are presented to the user.

The user interface 620 may also include a set of condition or parameterfields (not shown). These fields can be used to specify when certainaugmented reality items in a given collection are activated and theconditions for activating the augmented reality items (e.g.,event-based, location-based, and time-based conditions). Once the useris satisfied with the collections that are defined, the user can savethe collections and publish the collections to enables users to selectthe collections.

As an example, as shown in FIG. 7, a user interface 710 may be presentedthat lists various augmented reality item collections. A user can selecta given augmented reality item collection 712. In response, userinterface 720 is presented. User interface 720 includes an image orvideo of a person's face (this can be a real-time video or live video ora previously captured video or image). The augmented reality items 721associated with the augmented reality item collection are presentedusing icons in a menu.

In some implementations, user interface 720 presents a live camera feedfrom a mobile device or mobile phone together with an application iconfor each augmented reality item in the selected augmented reality itemcollection 712. A selected one of the augmented reality items presentedtogether with the live camera feed is applied dynamically to the camerafeed to add one or more graphical elements to one or more objectsdepicted in the camera feed. In certain embodiments, multiple augmentedreality items can be selected to be applied concurrently and incombination such that graphical elements from each of the multipleaugmented reality items are retrieved and applied simultaneously to oneor more objects depicted in the live camera feed. For example, alipstick augmented reality item and an eyeliner augmented reality itemcan both be selected from the augmented reality items 721 that arepresented with the live camera feed to modify (apply graphical elementsto) the lips and the eyes of a face depicted in the camera feed.

A first augmented reality item can be selected and attributes 722 arepresented for adjusting properties of the first augmented reality itemthat is selected. As an example, the first augmented reality item is alipstick style and the properties that can be adjusted are the colors ofthe lipstick style. As the user changes the color of the lipstick byselecting the attributes 722, the lips portion 724 of the face int ehimage or video is changed to have the style, properties or color of theselected lipstick style and color. In some cases, each attribute 722(e.g., color) is automatically presented for a threshold period of time(e.g., 4 seconds) after which a subsequent attribute 722 (e.g., color)is presented. After all of the attributes are automatically progressedthrough and presented, a second lipstick or makeup style 721 isautomatically selected and presented on the face depicted in the image.Attributes associated with the second lipstick or makeup style 721 areagain automatically progressed one at a time and activated and presentedfor the threshold period of time.

As another example, the user interface 720 is presented when a userselects an individual augmented reality item from a list of augmentedreality items shown in user interface 810 (FIG. 8A). In response toreceiving a user selection of the first augmented reality item 812 fromthe list shown in user interface 810, the other augmented reality itemsthat are in the same collection as the first augmented reality item 812are retrieved. The user interface 720 is then presented in which theselected first augmented reality item 812 is activated and the remainingaugmented reality items are displayed.

In some implementations, a user can subscribe to one or more augmentedreality item collections, such as by tapping on a subscribe iconassociated with a given augmented reality item collection. Bysubscribing to a given augmented reality item collection, the user canremain informed with any changes (e.g., additions or removals) ofaugmented reality items in the given augmented reality item collection.Whenever a change is detected, such as an augmented reality item beingadded to the augmented reality item collection, a notification, such asa push notification, is sent to the client device 102 which issubscribed to the collection. The push notification may include anoption to access or view the collection with the added augmented realityitem or view just the newly added augmented reality item. For example,the user can swipe on the notification to automatically launch a camerafeed, such as a live camera feed, and to automatically activate thenewly added augmented reality item identified by the notification.

As shown in FIG. 8A, a user interface 840 is presented for presenting alist of subscriptions the user has created to various augmented realityitem collections 842. To reach user interface 840, a user swipes rightor left from the user interface 810 one or more times. Each augmentedreality item collection 842 to which the user subscribes is representedby an icon or thumbnail that includes a badge or indicator 843. Theindicator 843 indicates the number of new augmented reality items thathave been added to the augmented reality item collection 842 since theuser last accessed the augmented reality item collection 842. In somecases, a given augmented reality item collection to which the usersubscribes expires or is made unavailable after a given interval or endtime. At that time, the icon or thumbnail corresponding to the expiredcollection is removed from user interface 840. If the user selects anunsubscribe option associated with a given collection, the icon orthumbnail corresponding to the unsubscribed collection is removed fromuser interface 840 and the user will no longer receive notificationsidentifying changes to the collection.

In response to receiving a user selection of the icon or thumbnailcorresponding to augmented reality item collection 842, a live camerafeed is presented to the user and the newest augmented reality item inthe collection is automatically presented and activated. When the userreturns to view the list of augmented reality item collections to whichthe user subscribes in user interface 840, the previously presentedbadge or indicator 843 is removed as the user has already viewed oraccessed the new augmented reality items represented by the indicator843.

FIG. 8B shows a user interface for accessing augmented reality items andaugmented reality item collections. Specifically, user interface 820 ispresented in which an augmented reality item collection 821 is shown atthe top of the screen using an icon or thumbnail. The icon or thumbnailmay include an animation corresponding to the augmented reality itemcollection 821. For example, the augmented reality item collection 821corresponds to superheroes or characters in a movie, comic book, video,or story. In this case, the icon or thumbnail may include a short 5second clip of the movie or video in which the characters or superheroesof the augmented reality item collection 821 appear. The user can selectan option to access the augmented reality items of the augmented realityitem collection 821, such as by tapping on the icon or thumbnailrepresenting the collection. In response, a camera feed 822 is presentedtogether with the list of augmented reality items available forselection from the collection 821. In some cases, the messaging clientapplication 104 processes a profile of the user to determine andidentify a favorite superhero of the user from the movie, comic book,video, or story. If a favorite superhero is found, the messaging clientapplication 104 automatically navigates to and activates the augmentedreality item corresponding to the favorite superhero in a camera feed inresponse to receiving the user selection of the icon or thumbnail forthe collection 821. To do so, the messaging client application 104retrieves an identifier (e.g., a superhero name) from the user's profileand searches metadata of the augmented reality items in the selectedcollection to identify an augmented reality item associated with theretrieved identifier. If one is found, the augmented reality item isautomatically activated when the camera feed is presented or is moved tothe top of the list of augmented reality items in the collection thatare presented to the user.

Camera feed 822 depicts a face of the user and when a user selects agiven augmented reality item from the list of augmented reality items831, visual properties, such as graphical elements associated with theselected augmented reality item are retrieved and used to modify theface depicted in the camera feed 822. For example, a helmet or face 832corresponding to the superhero represented by the selected augmentedreality item overlays or replaces the face depicted in the camera feed.

FIG. 9 is a block diagram illustrating an example software architecture906, which may be used in conjunction with various hardwarearchitectures herein described. FIG. 9 is a non-limiting example of asoftware architecture and it will be appreciated that many otherarchitectures may be implemented to facilitate the functionalitydescribed herein. The software architecture 906 may execute on hardwaresuch as machine 1000 of FIG. 10 that includes, among other things,processors 1004, memory 1014, and input/output (I/O) components 1018. Arepresentative hardware layer 952 is illustrated and can represent, forexample, the machine 1000 of FIG. 10. The representative hardware layer952 includes a processing unit 954 having associated executableinstructions 904. Executable instructions 904 represent the executableinstructions of the software architecture 906, including implementationof the methods, components, and so forth described herein. The hardwarelayer 952 also includes memory and/or storage modules memory/storage956, which also have executable instructions 904. The hardware layer 952may also comprise other hardware 958.

In the example architecture of FIG. 9, the software architecture 906 maybe conceptualized as a stack of layers where each layer providesparticular functionality. For example, the software architecture 906 mayinclude layers such as an operating system 902, libraries 920,frameworks/middleware 918, applications 916, and a presentation layer914. Operationally, the applications 916 and/or other components withinthe layers may invoke API calls 908 through the software stack andreceive messages 912 in response to the API calls 908. The layersillustrated are representative in nature and not all softwarearchitectures have all layers. For example, some mobile or specialpurpose operating systems may not provide a frameworks/middleware 918,while others may provide such a layer. Other software architectures mayinclude additional or different layers.

The operating system 902 may manage hardware resources and providecommon services. The operating system 902 may include, for example, akernel 922, services 924, and drivers 926. The kernel 922 may act as anabstraction layer between the hardware and the other software layers.For example, the kernel 922 may be responsible for memory management,processor management (e.g., scheduling), component management,networking, security settings, and so on. The services 924 may provideother common services for the other software layers. The drivers 926 areresponsible for controlling or interfacing with the underlying hardware.For instance, the drivers 926 include display drivers, camera drivers,Bluetooth® drivers, flash memory drivers, serial communication drivers(e.g., Universal Serial Bus (USB) drivers), Wi-Fi® drivers, audiodrivers, power management drivers, and so forth depending on thehardware configuration.

The libraries 920 provide a common infrastructure that is used by theapplications 916 and/or other components and/or layers. The libraries920 provide functionality that allows other software components toperform tasks in an easier fashion than to interface directly with theunderlying operating system 902 functionality (e.g., kernel 922,services 924 and/or drivers 926). The libraries 920 may include systemlibraries 944 (e.g., C standard library) that may provide functions suchas memory allocation functions, string manipulation functions,mathematical functions, and the like. In addition, the libraries 920 mayinclude API libraries 946 such as media libraries (e.g., libraries tosupport presentation and manipulation of various media format such asMPREG4, H.264, MP3, AAC, AMR, JPG, PNG), graphics libraries (e.g., anOpenGL framework that may be used to render two-dimensional andthree-dimensional in a graphic content on a display), database libraries(e.g., SQLite that may provide various relational database functions),web libraries (e.g., WebKit that may provide web browsingfunctionality), and the like. The libraries 920 may also include a widevariety of other libraries 948 to provide many other APIs to theapplications 916 and other software components/modules.

The frameworks/middleware 918 (also sometimes referred to as middleware)provide a higher-level common infrastructure that may be used by theapplications 916 and/or other software components/modules. For example,the frameworks/middleware 918 may provide various graphic user interfacefunctions, high-level resource management, high-level location services,and so forth. The frameworks/middleware 918 may provide a broad spectrumof other APIs that may be utilized by the applications 916 and/or othersoftware components/modules, some of which may be specific to aparticular operating system 902 or platform.

The applications 916 include built-in applications 938 and/orthird-party applications 940. Examples of representative built-inapplications 938 may include, but are not limited to, a contactsapplication, a browser application, a book reader application, alocation application, a media application, a messaging application,and/or a game application. Third-party applications 940 may include anapplication developed using the ANDROID™ or IOS™ software developmentkit (SDK) by an entity other than the vendor of the particular platform,and may be mobile software running on a mobile operating system such asIOS™, ANDROID™, WINDOWS® Phone, or other mobile operating systems. Thethird-party applications 940 may invoke the API calls 908 provided bythe mobile operating system (such as operating system 902) to facilitatefunctionality described herein.

The applications 916 may use built-in operating system functions (e.g.,kernel 922, services 924, and/or drivers 926), libraries 920, andframeworks/middleware 918 to create UIs to interact with users of thesystem. Alternatively, or additionally, in some systems, interactionswith a user may occur through a presentation layer, such as presentationlayer 914. In these systems, the application/component “logic” can beseparated from the aspects of the application/component that interactwith a user.

FIG. 10 is a block diagram illustrating components of a machine 1000,according to some example embodiments, able to read instructions from amachine-readable medium (e.g., a machine-readable storage medium) andperform any one or more of the methodologies discussed herein.Specifically, FIG. 10 shows a diagrammatic representation of the machine1000 in the example form of a computer system, within which instructions1010 (e.g., software, a program, an application, an applet, an app, orother executable code) for causing the machine 1000 to perform any oneor more of the methodologies discussed herein may be executed. As such,the instructions 1010 may be used to implement modules or componentsdescribed herein. The instructions 1010 transform the general,non-programmed machine 1000 into a particular machine 1000 programmed tocarry out the described and illustrated functions in the mannerdescribed. In alternative embodiments, the machine 1000 operates as astandalone device or may be coupled (e.g., networked) to other machines.In a networked deployment, the machine 1000 may operate in the capacityof a server machine or a client machine in a server-client networkenvironment, or as a peer machine in a peer-to-peer (or distributed)network environment. The machine 1000 may comprise, but not be limitedto, a server computer, a client computer, a personal computer (PC), atablet computer, a laptop computer, a netbook, a set-top box (STB), apersonal digital assistant (PDA), an entertainment media system, acellular telephone, a smart phone, a mobile device, a wearable device(e.g., a smart watch), a smart home device (e.g., a smart appliance),other smart devices, a web appliance, a network router, a networkswitch, a network bridge, or any machine capable of executing theinstructions 1010, sequentially or otherwise, that specify actions to betaken by machine 1000. Further, while only a single machine 1000 isillustrated, the term “machine” shall also be taken to include acollection of machines that individually or jointly execute theinstructions 1010 to perform any one or more of the methodologiesdiscussed herein.

The machine 1000 may include processors 1004, memory/storage 1006, andI/O components 1018, which may be configured to communicate with eachother such as via a bus 1002. In an example embodiment, the processors1004 (e.g., a central processing unit (CPU), a reduced instruction setcomputing (RISC) processor, a complex instruction set computing (CISC)processor, a graphics processing unit (GPU), a digital signal processor(DSP), an application-specific integrated circuit (ASIC), aradio-frequency integrated circuit (RFIC), another processor, or anysuitable combination thereof) may include, for example, a processor 1008and a processor 1012 that may execute the instructions 1010. The term“processor” is intended to include multi-core processors 1004 that maycomprise two or more independent processors (sometimes referred to as“cores”) that may execute instructions contemporaneously. Although FIG.10 shows multiple processors 1004, the machine 1000 may include a singleprocessor with a single core, a single processor with multiple cores(e.g., a multi-core processor), multiple processors with a single core,multiple processors with multiple cores, or any combination thereof.

The memory/storage 1006 may include a memory 1014, such as a mainmemory, or other memory storage, and a storage unit 1016, bothaccessible to the processors 1004 such as via the bus 1002. The storageunit 1016 and memory 1014 store the instructions 1010 embodying any oneor more of the methodologies or functions described herein. Theinstructions 1010 may also reside, completely or partially, within thememory 1014, within the storage unit 1016, within at least one of theprocessors 1004 (e.g., within the processor's cache memory), or anysuitable combination thereof, during execution thereof by the machine1000. Accordingly, the memory 1014, the storage unit 1016, and thememory of processors 1004 are examples of machine-readable media.

The I/O components 1018 may include a wide variety of components toreceive input, provide output, produce output, transmit information,exchange information, capture measurements, and so on. The specific I/Ocomponents 1018 that are included in a particular machine 1000 willdepend on the type of machine. For example, portable machines such asmobile phones will likely include a touch input device or other suchinput mechanisms, while a headless server machine will likely notinclude such a touch input device. It will be appreciated that the I/Ocomponents 1018 may include many other components that are not shown inFIG. 10. The I/O components 1018 are grouped according to functionalitymerely for simplifying the following discussion and the grouping is inno way limiting. In various example embodiments, the I/O components 1018may include output components 1026 and input components 1028. The outputcomponents 1026 may include visual components (e.g., a display such as aplasma display panel (PDP), a light emitting diode (LED) display, aliquid crystal display (LCD), a projector, or a cathode ray tube (CRT)),acoustic components (e.g., speakers), haptic components (e.g., avibratory motor, resistance mechanisms), other signal generators, and soforth. The input components 1028 may include alphanumeric inputcomponents (e.g., a keyboard, a touch screen configured to receivealphanumeric input, a photo-optical keyboard, or other alphanumericinput components), point-based input components (e.g., a mouse, atouchpad, a trackball, a joystick, a motion sensor, or other pointinginstrument), tactile input components (e.g., a physical button, a touchscreen that provides location and/or force of touches or touch gestures,or other tactile input components), audio input components (e.g., amicrophone), and the like.

In further example embodiments, the I/O components 1018 may includebiometric components 1039, motion components 1034, environmentalcomponents 1036, or position components 1038 among a wide array of othercomponents. For example, the biometric components 1039 may includecomponents to detect expressions (e.g., hand expressions, facialexpressions, vocal expressions, body gestures, or eye tracking), measurebiosignals (e.g., blood pressure, heart rate, body temperature,perspiration, or brain waves), identify a person (e.g., voiceidentification, retinal identification, facial identification,fingerprint identification, or electroencephalogram basedidentification), and the like. The motion components 1034 may includeacceleration sensor components (e.g., accelerometer), gravitation sensorcomponents, rotation sensor components (e.g., gyroscope), and so forth.The environmental components 1036 may include, for example, illuminationsensor components (e.g., photometer), temperature sensor components(e.g., one or more thermometer that detect ambient temperature),humidity sensor components, pressure sensor components (e.g.,barometer), acoustic sensor components (e.g., one or more microphonesthat detect background noise), proximity sensor components (e.g.,infrared sensors that detect nearby objects), gas sensors (e.g., gasdetection sensors to detection concentrations of hazardous gases forsafety or to measure pollutants in the atmosphere), or other componentsthat may provide indications, measurements, or signals corresponding toa surrounding physical environment. The position components 1038 mayinclude location sensor components (e.g., a GPS receiver component),altitude sensor components (e.g., altimeters or barometers that detectair pressure from which altitude may be derived), orientation sensorcomponents (e.g., magnetometers), and the like.

Communication may be implemented using a wide variety of technologies.The I/O components 1018 may include communication components 1040operable to couple the machine 1000 to a network 1037 or devices 1029via coupling 1024 and coupling 1022, respectively. For example, thecommunication components 1040 may include a network interface componentor other suitable device to interface with the network 1037. In furtherexamples, communication components 1040 may include wired communicationcomponents, wireless communication components, cellular communicationcomponents, Near Field Communication (NFC) components, Bluetooth®components (e.g., Bluetooth® Low Energy), Wi-Fi® components, and othercommunication components to provide communication via other modalities.The devices 1029 may be another machine or any of a wide variety ofperipheral devices (e.g., a peripheral device coupled via a USB).

Moreover, the communication components 1040 may detect identifiers orinclude components operable to detect identifiers. For example, thecommunication components 1040 may include Radio Frequency Identification(RFID) tag reader components, NFC smart tag detection components,optical reader components (e.g., an optical sensor to detectone-dimensional bar codes such as Universal Product Code (UPC) bar code,multi-dimensional bar codes such as Quick Response (QR) code, Azteccode, Data Matrix, Dataglyph, MaxiCode, PDF417, Ultra Code, UCC RSS-2Dbar code, and other optical codes), or acoustic detection components(e.g., microphones to identify tagged audio signals). In addition, avariety of information may be derived via the communication components1040, such as location via Internet Protocol (IP) geo-location, locationvia Wi-Fi® signal triangulation, location via detecting a NFC beaconsignal that may indicate a particular location, and so forth.

Glossary

“CARRIER SIGNAL” in this context refers to any intangible medium that iscapable of storing, encoding, or carrying transitory or non-transitoryinstructions for execution by the machine, and includes digital oranalog communications signals or other intangible medium to facilitatecommunication of such instructions. Instructions may be transmitted orreceived over the network using a transitory or non-transitorytransmission medium via a network interface device and using any one ofa number of well-known transfer protocols.

“CLIENT DEVICE” in this context refers to any machine that interfaces toa communications network to obtain resources from one or more serversystems or other client devices. A client device may be, but is notlimited to, a mobile phone, desktop computer, laptop, PDA, smart phone,tablet, ultra book, netbook, laptop, multi-processor system,microprocessor-based or programmable consumer electronics, game console,set-top box, or any other communication device that a user may use toaccess a network.

“COMMUNICATIONS NETWORK” in this context refers to one or more portionsof a network that may be an ad hoc network, an intranet, an extranet, avirtual private network (VPN), a local area network (LAN), a wirelessLAN (WLAN), a wide area network (WAN), a wireless WAN (WWAN), ametropolitan area network (MAN), the Internet, a portion of theInternet, a portion of the Public Switched Telephone Network (PSTN), aplain old telephone service (POTS) network, a cellular telephonenetwork, a wireless network, a Wi-Fi® network, another type of network,or a combination of two or more such networks. For example, a network ora portion of a network may include a wireless or cellular network andthe coupling may be a Code Division Multiple Access (CDMA) connection, aGlobal System for Mobile communications (GSM) connection, or other typeof cellular or wireless coupling. In this example, the coupling mayimplement any of a variety of types of data transfer technology, such asSingle Carrier Radio Transmission Technology (1×RTT), Evolution-DataOptimized (EVDO) technology, General Packet Radio Service (GPRS)technology, Enhanced Data rates for GSM Evolution (EDGE) technology,third Generation Partnership Project (3GPP) including 3G, fourthgeneration wireless (4G) networks, Universal Mobile TelecommunicationsSystem (UMTS), High Speed Packet Access (HSPA), WorldwideInteroperability for Microwave Access (WiMAX), Long Term Evolution (LTE)standard, others defined by various standard setting organizations,other long range protocols, or other data transfer technology.

“EPHEMERAL MESSAGE” in this context refers to a message that isaccessible for a time-limited duration. An ephemeral message may be atext, an image, a video, and the like. The access time for the ephemeralmessage may be set by the message sender. Alternatively, the access timemay be a default setting or a setting specified by the recipient.Regardless of the setting technique, the message is transitory.

“MACHINE-READABLE MEDIUM” in this context refers to a component, device,or other tangible media able to store instructions and data temporarilyor permanently and may include, but is not limited to, random-accessmemory (RAM), read-only memory (ROM), buffer memory, flash memory,optical media, magnetic media, cache memory, other types of storage(e.g., Erasable Programmable Read-Only Memory (EEPROM)) and/or anysuitable combination thereof. The term “machine-readable medium” shouldbe taken to include a single medium or multiple media (e.g., acentralized or distributed database, or associated caches and servers)able to store instructions. The term “machine-readable medium” shallalso be taken to include any medium, or combination of multiple media,that is capable of storing instructions (e.g., code) for execution by amachine, such that the instructions, when executed by one or moreprocessors of the machine, cause the machine to perform any one or moreof the methodologies described herein. Accordingly, a “machine-readablemedium” refers to a single storage apparatus or device, as well as“cloud-based” storage systems or storage networks that include multiplestorage apparatus or devices. The term “machine-readable medium”excludes signals per se.

“COMPONENT” in this context refers to a device, physical entity, orlogic having boundaries defined by function or subroutine calls, branchpoints, APIs, or other technologies that provide for the partitioning ormodularization of particular processing or control functions. Componentsmay be combined via their interfaces with other components to carry outa machine process. A component may be a packaged functional hardwareunit designed for use with other components and a part of a program thatusually performs a particular function of related functions. Componentsmay constitute either software components (e.g., code embodied on amachine-readable medium) or hardware components. A “hardware component”is a tangible unit capable of performing certain operations and may beconfigured or arranged in a certain physical manner. In various exampleembodiments, one or more computer systems (e.g., a standalone computersystem, a client computer system, or a server computer system) or one ormore hardware components of a computer system (e.g., a processor or agroup of processors) may be configured by software (e.g., an applicationor application portion) as a hardware component that operates to performcertain operations as described herein.

A hardware component may also be implemented mechanically,electronically, or any suitable combination thereof. For example, ahardware component may include dedicated circuitry or logic that ispermanently configured to perform certain operations. A hardwarecomponent may be a special-purpose processor, such as aField-Programmable Gate Array (FPGA) or an ASIC. A hardware componentmay also include programmable logic or circuitry that is temporarilyconfigured by software to perform certain operations. For example, ahardware component may include software executed by a general-purposeprocessor or other programmable processor. Once configured by suchsoftware, hardware components become specific machines (or specificcomponents of a machine) uniquely tailored to perform the configuredfunctions and are no longer general-purpose processors. It will beappreciated that the decision to implement a hardware componentmechanically, in dedicated and permanently configured circuitry, or intemporarily configured circuitry (e.g., configured by software) may bedriven by cost and time considerations. Accordingly, the phrase“hardware component” (or “hardware-implemented component”) should beunderstood to encompass a tangible entity, be that an entity that isphysically constructed, permanently configured (e.g., hardwired), ortemporarily configured (e.g., programmed) to operate in a certain manneror to perform certain operations described herein. Consideringembodiments in which hardware components are temporarily configured(e.g., programmed), each of the hardware components need not beconfigured or instantiated at any one instance in time. For example,where a hardware component comprises a general-purpose processorconfigured by software to become a special-purpose processor, thegeneral-purpose processor may be configured as respectively differentspecial-purpose processors (e.g., comprising different hardwarecomponents) at different times. Software accordingly configures aparticular processor or processors, for example, to constitute aparticular hardware component at one instance of time and to constitutea different hardware component at a different instance of time.

Hardware components can provide information to, and receive informationfrom, other hardware components. Accordingly, the described hardwarecomponents may be regarded as being communicatively coupled. Wheremultiple hardware components exist contemporaneously, communications maybe achieved through signal transmission (e.g., over appropriate circuitsand buses) between or among two or more of the hardware components. Inembodiments in which multiple hardware components are configured orinstantiated at different times, communications between such hardwarecomponents may be achieved, for example, through the storage andretrieval of information in memory structures to which the multiplehardware components have access. For example, one hardware component mayperform an operation and store the output of that operation in a memorydevice to which it is communicatively coupled. A further hardwarecomponent may then, at a later time, access the memory device toretrieve and process the stored output.

Hardware components may also initiate communications with input oroutput devices and can operate on a resource (e.g., a collection ofinformation). The various operations of example methods described hereinmay be performed, at least partially, by one or more processors that aretemporarily configured (e.g., by software) or permanently configured toperform the relevant operations. Whether temporarily or permanentlyconfigured, such processors may constitute processor-implementedcomponents that operate to perform one or more operations or functionsdescribed herein. As used herein, “processor-implemented component”refers to a hardware component implemented using one or more processors.Similarly, the methods described herein may be at least partiallyprocessor-implemented, with a particular processor or processors beingan example of hardware. For example, at least some of the operations ofa method may be performed by one or more processors orprocessor-implemented components. Moreover, the one or more processorsmay also operate to support performance of the relevant operations in a“cloud computing” environment or as a “software as a service” (SaaS).For example, at least some of the operations may be performed by a groupof computers (as examples of machines including processors), with theseoperations being accessible via a network (e.g., the Internet) and viaone or more appropriate interfaces (e.g., an API). The performance ofcertain of the operations may be distributed among the processors, notonly residing within a single machine, but deployed across a number ofmachines. In some example embodiments, the processors orprocessor-implemented components may be located in a single geographiclocation (e.g., within a home environment, an office environment, or aserver farm). In other example embodiments, the processors orprocessor-implemented components may be distributed across a number ofgeographic locations.

“PROCESSOR” in this context refers to any circuit or virtual circuit (aphysical circuit emulated by logic executing on an actual processor)that manipulates data values according to control signals (e.g.,“commands,” “op codes,” “machine code,” etc.) and which producescorresponding output signals that are applied to operate a machine. Aprocessor may, for example, be a CPU, a RISC processor, a CISCprocessor, a GPU, a DSP, an ASIC, a RFIC or any combination thereof. Aprocessor may further be a multi-core processor having two or moreindependent processors (sometimes referred to as “cores”) that mayexecute instructions contemporaneously.

“TIMESTAMP” in this context refers to a sequence of characters orencoded information identifying when a certain event occurred, forexample giving date and time of day, sometimes accurate to a smallfraction of a second.

Changes and modifications may be made to the disclosed embodimentswithout departing from the scope of the present disclosure. These andother changes or modifications are intended to be included within thescope of the present disclosure, as expressed in the following claims.

What is claimed is:
 1. A method comprising: causing a messagingapplication to present a subset of augmented reality items; causing themessaging application to modify an image based on a first augmentedreality item in the subset; determining that the image has been modifiedby the first augmented reality item for a threshold period of time; andin response to determining that the image has been modified by the firstaugmented reality item for the threshold period of time, automaticallyreplacing the first augmented reality item with a second augmentedreality item in the subset to modify the image based on the secondaugmented reality item, wherein additional augmented reality items inthe subset are automatically applied to modify the image upon of eachsubsequent passage of the threshold period of time.
 2. The method ofclaim 1, further comprising: receiving input that selects a collectionof augmented reality items; obtaining an identifier of the collection ofaugmented reality items; searching, based on the identifier, a pluralityof augmented reality items to identify the subset of augmented realityitems associated with the identifier.
 3. The method of claim 1, furthercomprising: after automatically modifying the image based on the secondaugmented reality item, determining that the second augmented realityitem is a last augmented reality item in the subset; and in response todetermining that the image has been modified by the second augmentedreality item for the threshold period of time and that the secondaugmented reality item is the last augmented reality item in the subset,automatically replacing the second augmented reality item with a thirdaugmented reality item in the subset that precedes the first augmentedreality item.
 4. The method of claim 1, further comprising: afterautomatically modifying the image based on the second augmented realityitem, determining that the second augmented reality item is a lastaugmented reality item in the subset; and in response to determiningthat the image has been modified by the second augmented reality itemfor the threshold period of time and that the second augmented realityitem is the last augmented reality item in the subset, automaticallypresenting a message indicating that all augmented reality items in thesubset have been presented, the message comprising an option to repeatautomatic presentation of the subset of augmented reality items.
 5. Themethod of claim 1, further comprising: obtaining metadata associatedwith each of the augmented reality items in the subset; retrieving apresentation order from the metadata; and automatically applying thesubset of augmented reality items based on the presentation order. 6.The method of claim 1, further comprising: obtaining metadata associatedwith each of the augmented reality items in the subset; determining,based on the metadata, a condition for applying the second of theaugmented reality items after the first augmented reality item; andapplying the second augmented reality item in response to determiningthat the condition has been satisfied.
 7. The method of claim 6, whereinthe condition comprises sending the image modified based on the firstaugmented reality item from a first client device to a second clientdevice or capturing content that depicts a real-world object associatedwith the first augmented reality item.
 8. The method of claim 6, whereinthe condition comprises determining that a client device on which theimage is modified is within a threshold distance of a geographicallocation specified in the metadata.
 9. The method of claim 1, furthercomprising: obtaining a second subset of augmented reality items;obtaining a first index position associated with the first subset and asecond index position associated with the second subset; and arrangingidentifiers of the first and second subsets in a display of themessaging application based on the first and second index positions. 10.The method of claim 9, wherein the identifiers of the first and secondsubsets comprise thumbnails that represent respective augmented realityitems of the first and second subsets.
 11. The method of claim 1,further comprising: determining that a current time is within an activeperiod of the subset; and in response to determining that the currenttime is within the active period of the subset, presenting an indicatorof the subset in a display of the messaging application.
 12. The methodof claim 11, further comprising: removing the indicator of the subsetafter the active period of the subset ends.
 13. The method of claim 1,further comprising: prior to presenting the subset of augmented realityitems, receiving input that selects the first augmented reality itemassociated with the subset; and in response to receiving the input thatselects the first augmented reality item, obtaining an identifier of thesubset of augmented reality items.
 14. The method of claim 1, whereinthe subset comprises a makeup collection associated with a third-partyrelative to a provider of the messaging application, wherein the subsetof augmented reality items comprises a plurality of lipstick colors,further comprising: detecting a face in the image; identifying a lipsregion in the face; and applying a color associated with a first of theplurality of lipstick colors to the lips region of the face.
 15. Themethod of claim 14, further comprising: retrieving a catalog identifierand a product identifier of the first lipstick color; retrieving, from acomputing device of the third party, color information and priceinformation using the catalog identifier and product identifier; andpresenting the price information on the image while applying the colorbased on the retrieved color information.
 16. The method of claim 1,wherein the subset comprises a superheroes collection associated with athird-party relative to a provider of the messaging application, whereinthe subset of augmented reality items comprises a plurality ofsuperheroes, further comprising: detecting a person in the image; andmodifying the person based on attributes of a first superhero of theplurality of superheroes.
 17. The method of claim 1, further comprising:causing the messaging application to present icons of each of the subsetof augmented reality items; loading information associated withgenerating a display of a portion of the subset of the augmented realityitems; detecting input that navigates to select an icon of the iconsassociated with a second augmented reality item for which informationhas not yet been loaded; and loading the information for the secondaugmented reality item in response to detecting input that navigates toselect the icon associated with the second augmented reality item. 18.The method of claim 1, wherein the subset is a first collection, furthercomprising: causing the messaging application to present a first set oficons representing a plurality of different collections of augmentedreality items, the plurality of different collections comprising thefirst collection; receiving input that selects a first icon in the firstset of icons associated with the first collection of augmented realityitems; and causing the messaging application to present a second set oficons together with the first set of icons, the second set of iconsrepresenting the subset of augmented reality items of the firstcollection.
 19. A system comprising: a processor of a computing deviceconfigured to perform operations comprising: causing a messagingapplication to present a subset of augmented reality items; causing themessaging application to modify an image based on a first augmentedreality item in the subset; determining that the image has been modifiedby the first augmented reality item for a threshold period of time; andin response to determining that the image has been modified by the firstaugmented reality item for the threshold period of time, automaticallyreplacing the first augmented reality item with a second augmentedreality item in the subset to modify the image based on the secondaugmented reality item, wherein additional augmented reality items inthe subset are automatically applied to modify the image upon of eachsubsequent passage of the threshold period of time.
 20. A non-transitorymachine-readable storage medium that includes instructions that, whenexecuted by one or more processors of a machine, cause the machine toperform operations comprising: causing a messaging application topresent a subset of augmented reality items; causing the messagingapplication to modify an image based on a first augmented reality itemin the subset; determining that the image has been modified by the firstaugmented reality item for a threshold period of time; and in responseto determining that the image has been modified by the first augmentedreality item for the threshold period of time, automatically replacingthe first augmented reality item with a second augmented reality item inthe subset to modify the image based on the second augmented realityitem, wherein additional augmented reality items in the subset areautomatically applied to modify the image upon of each subsequentpassage of the threshold period of time.